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SUKHMY  AND  CONCLUSIO?«S 


Froblem 

The  CIC  Watch  Officer  has  a  sajor  responsibility  ior  the  receipt* 
Ir.tsrpretatioa,  and  trans3iis>siioa  of  tactical  coaHnunicctioBs  during  his 
wtcho  To  carry  out  v'  ■*«  responsibility  he  auf.  Ye  able  to  r&cogr.i2e 
and  understand  tactical  sigtiale  contained  in  the  Allied  Nava  Si;gsal 
Book.  The  purpose  of  this  reseaich  vss  to  evaluate  oral  progravswsed  in¬ 
struction  used  "ith  a  muititrack  tape  recorder  (the  Audio  Notebook)  as 
a  means  of  protacting  flexibility  in  Allied  Sava'*  Signal  Book  IffStruction 
and  better  adaption  to  individual  student  differences;  particularly 
thcae  differences  in  experience  level. 

an ^ Requirements 

The  student  body  for  the  CIC  Watch  Officer  Course  (K-2G-351)  at  the 
Fieec  Ar.ti-Alr  Warfare  Training  Center  {FA4WTRACENS0)  varies  from  petty 
r_ud  coiKi.iiBsioned  officers  with  as  msch  s@  15  years  SKpsrlence  to  newly 
ccmmissioned  ensigns.  The  leeture/claasroom  uoiil  program  presently 
't^ed  in  Allied  Signal  Book  inetruction  is  scarcely  needed  by  some  in 
huch  a  student  body  and  does  not.  permit  proper  pacing  or  sufficien*' 
practice  for  others.  To  cope  with  such  individual  differences  an  oral 
program  was  developed  for  J.earaijig  the  Allied  Naval  Sipval  Book.  Such 
a  program  not  only  takes  advantage  of  the  adaption  to  dlflerencea  in 
experience  as  well  as  to  individual  learning  rates,  but  It  also  intro¬ 
duces  flexibility  into  Instruction  by  pemittiug  Inatructora  to  work 
?rith  individuals  or  small  groups  for  other  purposes  while,  most  studients 
arc  ieaiTiing  the  Signal  Book  procedures  from  the  program. 

roach 


the  program  was  written  Co  achieve  the  tactical  cogBnunicaticns  l.earaiag 
objectives  as  stated  in  the  recently  redesigned  curriculum  for  the 
FAAWTMCENSD  CICli\fO  course.  Allied  'Naval  Signal  Book  procedures  ware 
taught  to  student  officers  utilising  the  oral  learning  program.  Their 
achievemeiic  was  comoared  with  groups  taught  by  the  lecture/classroom 
drill  method.  Data  jn  time  required  were  obtained.  Student  attiCudns 
Toward  the  program  ware  solicited. 

?i.aJJ.r.gs,  Concluslona,  and  Recoiamendstions 

1.  Ttie  objectlvee  of  the  Signel  Book  esgaent  of  the  cuj,.  ..cul»*«  are 
significantly  better  using  the  ortl  leaiAlng  program,  (kes* 

2.  Ihe  student  controlled  prc-graaatBd  learning  takes  leai  l!^  than  the 
cO£iVentio.;al  lecture  for  all  but  ;ha  very  alcwett  student. 

3  and  11) 

3.  1^19  ebove  findings  roeultod  In  Immediate  incoirporatlo:;  of  the 

progriiK  into  th^  CICWO  course.  (Page  iC) 
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of  Naval  i’ersomiel  in  I'uture  plaiuiing,  it  is  requested  that  the  use 
and  evjiluatiou  form  on  the  reverse  of  this  page  be  completed  and 
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with  tape,  and  nail. 
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PROGRAMMED  INSTRUCTION  FOR  SELECTED  CIC  WATCH  OFFICER  TASKS: 
II.  AN  EXPERIMENTAI.  EVALUATION  OF  THE  AUDIO  NOTEBOOK 
IN  THE  TEACHING  OF  THE  ALLIED  NAVAL  SIGNAL  BOOK 


A.  Introduction 

The  Allied  Naval  Signal  Book  is  a  confidential  publication  which  is 
widely  used  for  fleet  tactical  conununications.  Whether  a  signal  is  sent 
by  flaghoist  or  radiotelephone,  the  Combat  Information  Center  Watch  Officer 
(CICWO)  has  the  responsibility  of  encoding  or  decoding,  or  monitoring  the 
encoding  or  decoding  of  every  signal  which  his  ship  either  originates  or 
receives  while  he  is  on  watch.  Not  only  dees  CIC  support  the  bridge  by  so 
doing,  but  the  tactical  picture  in  CIC  is  kept  current  by  accurate  use  of 
the  Signal  Book.  It  is  essential  that  the  CICWO  be  intimately  familiar 
with  the  Allied  Naval  Signal  Book  and  other  tactical  signaling  publications. 

In  the  past  the  Allied  Naval  Signal  Book  has  been  taught  in  the  CICWO 
course  at  Fleet  Anti-Air  Warfare  Training  Center,  San  Diego  (FAAVTiRACENSD) , 
by  the  lecture  classroom  drill  nethod.  An  Instructor  lectured  on  the  pub¬ 
lication  for  three  fifty-minute  periods;  each  student  had  one  copy  oi  the 
Signal  Book  and  as  the  instructor  made  his  demonstration  the  student  fol¬ 
lowed  along  as  best  he  could.  A  final  50  minute.-!  were  spent  in  a  drill; 
each  student  received  2  handout  of  signals  to  be  decoded  and  encoded.  These 
handouts  were  then  reviewed  in  the  classroom  by  the  instructor  for  approxi¬ 
mately  15  minutes.  Evaluation  of  a  student's  ability  consisted  partly  of 
instructor  judgment  of  his  performance  on  this  drill  and  in  a  four-nour 
mock-up  later  in  the  course,  bat  largely  by  a  series  of  questions  about  the 
Signal  Book  on  an  objective,  written  examination.  The  first  two  methods, 
at  best,  gave  each  student  three  opportunities  to  be  raeasureo  on  his  use  of 
the  Allied  Naval  Signal  Book.  The  latter  method  tested  not  their  skill  in 
the  use  of  the  book  but  rather  their  knowledge  about  the  book.  Howevei, 
based  on  the  mock-up  and  classroom  performances,  the  training  procedure  was 
considered  to  be  producing  a  satisfactory  level  of  achievement  in  the  use 
of  the  Allied  Naval  Signal  Book. 


One  problem  has  been  consistently  recognized  in  conducting  the  instruc¬ 
tion  for  the  COMTRAPAC  course,  K-2G-351,  "The  CIC  Watch  Officer":  the  wide 
variation  in  student  input.  Under  an  ongoing  program  of  redesigning 
K-2G-351,  stress  is  being  placed  on  providing  means  of  adapting  instruction 
to  this  wide  range  of  experience  in  the  student  body.  One  way  of  doing 
this  is  to  program  some  of  the  subject  matter  now  being  taught  by  lectures. 

The  Allied  Naval  Signal  Book  was  chosen  for  programming  for  three 
reasons:  (1)  a  previous  oral  programming  of  radiotelephone  procedures  had 

proved  successful  (Curran  and  Brock,  1967),  (2)  experience  in  the  use  of 
the  Signal  Book  among  the  CICWO  students  was  extremely  varied,  and  (3)  using 
the  Signal  Book  is  both  an  Intellectual  and  nianual  task  which  requires  both 
hands  of  the  user  to  be  free.  The  Audio  Notebook^,  a  miniaturized  multitrack 


A  product  of  Electronic  Futures,  Inc.,  North  Haven,  Connecticut. 


tape  recorder  which  provides  volume  storage  and  selective  playback  for 
learning  and  practice  of  subject  matter,  meets  the  third  criterion.  It  has 
earphones  for  the  student  and  requires  a  minimum  of  manipulation. 

A  major  advantage  of  the  Audio  Notebook  is  its  capability  for  branching 
to  and  from  the  22  15-mlnute  channels  on  the  tape.  Iftiile  the  Signal  Book 
novice  may  listen  to  all  roaterial  on  every  channel  of  the  program,  the 
experienced  student  can  branch  around  certain  basic  information  which  he 
already  knows.  The  time  required  for  the  experienced  student  to  cover  a 
certain  topic  can  therefore  be  greatly  reduced.  This  contrasts  sharply  with 
the  instructional  method  formerly  employed,  where  each  student  and  an  in¬ 
structor  were  tied  to  a  common  time  frame  which  was  too  short  for  the  inex¬ 
perienced  student  to  achieve  the  objectives  and/or  longer  than  necessary  for 
the  experienced  man  to  achieve  the  same  objectives. 

It  is  emphasized  that  it  is  not  only  to  provide  opportunity  for  the 
student  to  begin  where  his  previous  experience  has  brought  him  and  to  pace 
hir  own  instruction  that  the  Signal  Book  was  selected  for  programming; 
properly  programmed  an  Audio  Notebook  can  be  effectively  operated  by  a 
student  independently  of  Instructor  assistance.  The  latter  factor  permits 
great  flexibility  in  dealing  with  only  part  of  the  class  or  with  single 
individuals.  While  some  students  are  Involved  with  the  Signal  Book  program, 
others  can  be  receiving  remedial  or  advanced  training  from  other  programs 
or  from  newly  freed  instructors. 

This  study  was  Intended  to  accomplish  a  dual  purpose:  (1)  to  develop 
an  effective  program  for  teaching  the  use  of  the  Allied  Naval  Signal  Book 
for  the  CIC  Watch  Officer  and  (2)  to  test  the  suitability  of  the  Audio  Note¬ 
book  itself  for  use  by  FAAWTRACENSD  and  other  training  and  operational 
commands  as  a  means  of  promoting  flexibility  in  instruction.  In  the  event 
the  Audio  Notebook  proved  satlsfacLory,  the  program  could  be  introduced  into 
the  course. 


B.  Procedure 


1.  Experimental  Design 

This  study  was  carried  out  in  the  context  of  ongoing  CIC  Watch  Officer 
courses  at  FMWTRACENSD.  The  students  in  the  classes  convening  on  the 
following  dates  were  the  control  group  for  the  experiment;  they  received 
the  normal  lecture:  13  May  1968,  24  June  1968,  8  July  1968,  22  July  1968, 

5  August  1968,  and  6  October  1968.  There  were  a  total  of  114  control 
subjects.  The  experimental  group  of  students,  who  attended  the  classes 
convening  on  12  November  1968  and  25  November  1968,  received  the  Audio 
Notebook  program  plus  a  50-minute  classroom  drill.  There  were  29  subjects 
in  the  experimental  group. 

Every  subject  received  a  pretest  and  a  post-test,  both  of  which  pre¬ 
sented  the  student  with  five  signals  to  decode  and  four  signals  to  encode. 
The  difference  between  the  post-test  scores  of  the  control  and  experimental 
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groups,  with  pretost  scores  taken  into  account,  was  considered  the  measure 
of  the  difference  in  learning.  Maxlmxun  score  on  both  tests  was  90. 

Students  using  the  Audio  Notebook  were  given  a  questionnaire  on,  among 
other  things,  the  Allied  Naval  Signal  Book  program.  Time  samples  were  also 
obtained  from  notebook  users  so  as  to  compare  their  time  to  learn  with 
students  receiving  tl.'e  150  minutes  of  lecture. 

2.  Description  of  Learning  Program 

In  order  to  compare  directly  the  results  of  the  programmed  instruction 
group  with  the  lecture  group,  the  content  of  the  Audio  Notebook  program  was 
essentially  the  same  as  that  of  the  lecture.  Using  the  lesson  plan  of  the 
classroom  lecture  as  a  general  guideline,  one  former  naval  officer,  expert 
in  the  use  of  the  Allied  Naval  Signal  Book  and  also  knowledgeable  in  the 
principles  of  programmed  instruction,  planned  and  developed  the  program  for 
the  Audio  Notebook. 

This  program  consisted  of  ten  channels  of  material  covering  approximate¬ 
ly  two  hours  of  actual  tape  time.  The  general  outline  of  the  presentation 
is  given  in  Figure  1  and  Table  1.  The  students  received  a  brief  lecture  on 
administrative  procedures,  such  as  what  tc  do  when  finished  x^lth  the  program, 
vjhen  to  take  coffee  breaks,  etc.,  and  were  then  told  to  start  the  machine 
and  receive  further  instructions  on  channel  one  of  the  program. 

The  primary  difference  between  the  Audio  Notebook  program  and  the 
lectures  received  by  the  control  group  was  that  the  former  (1)  permitted 
the  student  to  take  as  much  time  as  needed  to  locate  a  particular  signal 
before  moving  on  into  the  program;  and  (2)  provided  immediate  feedback  on 
every  signal  in  the  program  after  presenting  a  signal  to  decode  and  a  suit¬ 
able  time  delay. 


C.  Results 


Means  and  standard  deviations  for  the  pretest  and  post-test  scores  for 
the  two  groups  are  shown  in  Table  2.  An  analysis  of  covariance  shows  the 
difference  between  the  two  mean  scores  is  statistically  significant 
(Table  3).  Because  of  the  small  N  in  the  experimental  group,  25  students 
from  each  group  were  matched  on  pretest  scores.  Again,  the  experimental 
group  is  significantly  better  than  the  control  group  (Table  4). 

Students  who  received  the  Audio  Notebook  program  but  were  not  used  in 
the  above  analysis  were  asked  to  rate  the  program  on  a  six  point  scale. 

The  ratings  and  number  of  selections  for  each  are  shown  in  Table  5.  Of  the 
83  students  responding,  68  gave  positive  ratings.  The  students  were  also 
asked  to  rate  the  Audio  Notebook  as  a  teaching  device  using  the  standard 
Navy  1.0  -  4.0  scale.  The  results  of  this  study  are  shown  in  Table  6.  The 
mean  rating  was  3.47  with  only  five  unsatisfactory  ratings. 

Additional  data  were  collected  to  determine  the  amount  of  time  spent 
in  the  program.  One  student  completed  the  program  in  55  minutes.  However, 
most  students  fell  in  a  range  between  90  minutes  and  150  minutes.  A  minimum 
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ALL  STUDEMTS  S?'" 


OPTIONAL 


MINUTES 


Fig.  1.  Diagram  of  the  Allied  Naval  Signal  Book  Audio  Notebook  Program. 
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TABLE  1 


Outline  of  the  Allied  Naval  Signal  Book  Audio  Notebook  Learning  Program 


Chan¬ 

nel 

Subject 

Maximum 

Track 

Time 

Minimum 

Track 

Time 

Remarks 

1 

Introduction  to  the 
Audio  Notebook 

4 '55" 

1'15" 

Most  students  will  already 
have  used  the  Audio  Notebook 

2 

Introduction  to  the 
Allied  Naval  Signal 

Book 

14'55" 

14 '30" 

Five-question  test  follows 
introductory  material. 

3 

Special  pennants 

10 '00" 

10 '00" 

If  questions  on  channel  2  are 
all  answered  correctly,  this 
channel  is  skipped. 

4 

Pre-test  for  funda¬ 
mentals 

12 '13" 

9'15" 

15  signals  are  presented  which 
student  must  evaluate  as  good 
or  bad.  No  misses:  go  to 
channel  8;  depending  upon  which 
of  the  signals  is  misjudged, 
student  is  sent  to  channels 

5,6,  or  7  respectively. 

5 

Basic  groups  and 
numerical  suffixes 

10'25'' 

8  *40" 

After  instruction  is  given, 
student  receives  same  direc¬ 
tion  as  on  channel  4. 

6 

TACK,  the  Addenda, 
and  letter  suffixes 

15 '00" 

6 ’35" 

Same  as  for  channel  5. 

7 

Governing  groups, 
governing  pennants, 
and  special  pennants 

15 '00" 

15'00 

Same  as  for  channel  5. 

8 

The  One  and  Nine  Tables 
with  Pre-test 

10'05" 

3'55" 

Four  signal  pre-test.  If  no 
errors,  student  goes  on  to 
channel  9. 

9 

The  yellow  tabbed 
signals 

11 '20" 

11 '20" 

10 

Encoding 

5 '30" 

1 ' 35" 

Two  signal  pre-test.  If 
both  encoded,  program  ends. 

7 
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TABLE  2 


Comparison  of  Allied  Naval  Signal  Book  Scores 


Control  Group  Experimental  Caoup 

(N=114)  (N=29) 


Raw 

Score 

Percentage 

Raw 

Score 

Percentage 

Post-Test  Mean 

65.08 

72.3 

77.9 

86.5  p  =.001 

Post-Tes*-  Standard 
Deviation 

14.1 

15.5 

Pretest  Mean 

34.8 

38.6 

35.4 

39.3  ns 

Pretest  Standard 
Deviation 

10.1 

13.3 

TABLE  3 

Analysis  of 

Covariance 

for  Pretest 

and  Post- 

-Test  Scores 

Source 

SS 

df 

MS 

F 

Treatments 

3627.21 

X 

3627.21 

19.666  p  /  .001 

Error 

2.5,821.47 

140 

184  '‘4 

TABLE  4 

Comparison  of  Post-Test  Scores 
(N*>25  pairs) 

for  Matched 

Pairs 

Control  Croup 

Experimental  Group 

Raw  Percentage 

Score 

Raw 

Score 

Percentage 

Mean 

63.0  70.0 

77.0 

00 

A 

Standard  Deviation 

17.0 

i6 . 3 

Pretest  Mean: 

34.0 

34.0 

Pretest  St.andard 

Deviation: 

12.6 

12.6 

TABLE  5 


Student  Rating  of  the  Allied  Naval  Signal 
Book  Audio  Notebook  Program 


Rating  No. 

of 

_  Students 


1  Highly  superior  to  conventional  instruction  19 

2  Somewhat  superior  to  conventional  instruction  21 

3  Better  than  conventional  instruction  28 

A  As  good  as  conventional  instruction  11 

5  Not  as  good  as  conventional  instruction  3 

6  Much  worse  than  conventional  Instruction  1 

TOTAL  83 


TABLE  6 

Student  Ratings  of  the  Audio  Notebook 


Rating  4,0 

3.8 

3.6 

3.4 

3.2 

3.0 

2.8 

2.6 

No,  of  students  10 

16 

20 

8 

6 

10 

4 

4 

Percent  of  students  12 

19 

24 

10 

7 

12 

5 

5 

of  one  classroom  hour  savings,  or  40  percent,  was  apparent  for  ail  but  the 
slowest  students.  The  distribution  of  times  is  shown  in  Figure  2,  with  the 
three-classroom-hour  lecture  shown  for  comparative  purposes . 

D.  Discussion 


As  Indicated  by  the  findings  reported  in  the  preceding  section,  the 
Audio  Notebook,  in  this  case  supplemented  by  the  classroom  drill,  was  found 
useful  for  increasing  the  flexibility  of  instruction.  The  improvement  in 
performance  of  the  students  who  learned  Signal  Book  procedures  by  tb  . 
method  as  compared  to  the  traditional  classroom  lecture  method,  and  the 
shorter  time  required  in  most  cases  to  achieve  the  same  objectives,  indi¬ 
cates  the  advisability  of  expanding  the  use  of  this  device  to  other  in¬ 
struction  where  the  stimuli  are  oral,  e.g.,  plotting  tasks  or  sound  powered 
phone  talker  procedures.  The  simplicity,  reliability,  ^nd  portability  of 
the  Notebook,  and  the  favorable  student  reaction  to  the  realism  provided 
make  it  extremely  u-eful  for  the  teaching  of  such  topics. 

The  time  saving  deserves  further  comment.  With  the  additional  time 
available,  the  student  can  either  go  to  other  learning  programs  or  drill 
furthei  on  the  signal  book.  The  adjustment  of  the  program  to  the  students' 
Individual  differences  seems  the  primary  advantage  of  this  program. 

Used  along  ''ith  various  written  programs  and  individualized  training 
exercises,  the  Audio  Notebook  Allied  Naval  Signal  Book  program  has  become 
a  regular  feature  of  FAAWTRACENSD  course  K-2G-351. 
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